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Introduction 

The modern theory of elhptic boundary value problems deals with two types of prob- 
lems. First, one has classical boundary value problems that admit a realization as some 
bounded operators in Sobolev spaces. Secondly, there are boundary value problems of 
Atiyah-Patodi-Singer type that can be realized as Fredholm operators in some subspaces 
of the Sobolev spaces. Moreover, the subspaces in question are the ranges of some pseu- 
do differential projections acting in the Sobolev spaces. These two classes of boundary 
value problems have a substantial difference. Namely, the first class is defined only for 
some elliptic operators (defined on manifolds with boundary) and the corresponding ob- 
struction in the stable theory is known as the Atiyah-Bott condition The second 
class of boundary value problems does not have this restriction: Fredholm boundary 
value problems of the above described type can be defined for an arbitrary elliptic opera- 
tor. On the other hand, this type of boundary conditions imposes an essential restriction 
on the right-hand sides of the boundary value problem. Namely, it is supposed that the 
right-hand sides are taken from a subspace of the Sobolev space of a possibly infinite codi- 
mension. The following question naturally arises: is it possible to construct an elliptic 
theory, which is a deformation of the two theories such that on the one end it reduces 
to the theory of classical boundary value problems, while at the other end it gives the 
Atiyah-Patodi-Singer problems? In other words, the problem is to construct a series of 
intermediate theories of elliptic boundary value problems that would contain as particular 
(and in a sense polar) examples classical boundary value problems and the problems with 
projections. The present paper is devoted to this problem. 

Clearly, the construction of such a theory requires some additional assumptions on 
the geometry of the boundary of the manifold, where the boundary value problems are 
considered. In the present paper we assume that the boundary is a fibration over a 
compact base with a compact fiber. In this setting, we establish all the analytical results 
we need (finiteness theorem, in particular). Moreover, we compute the obstruction to the 
existence of elliptic problems of this type. The obstruction turns out to be an element of 
i^^-group of the cotangent bundle of the base. This explains the nature of the topological 
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Atiyah-Bott obstruction as well as the absence of obstruction for Atiyah-Patodi-Singer 
problems. Indeed, in the special case, when the base coincides with the boundary, one 
obtains classical boundary value problems. On the contrary, when the base of the fibration 
is a point, we obtain Atiyah-Patodi-Singer problems. 

It should be mentioned that the importance and the interest of this class of boundary 
value problems on manifolds with fibered boundary is clear even in the case, when the 
boundary is a covering, i.e. a fibration with a discrete fiber. For a covering, the class 
of boundary value problems under consideration includes a number of nonlocal boundary 
value problems (see [Q). Rather surprisingly, the boundary conditions in the intermediate 
theories on manifolds with fibered boundary are defined by operators with discontinuous 
symbols. 

Let us briefly describe the contents of the paper. We construct an algebra of opera- 
tors with discontinuous symbols on a fibration in the first three sections. In particular, 
we establish the composition formula and prove the Fredholm criterion. Boundary value 
problems on manifolds with fibered boundary are defined in Section ^. We prove the finite- 
ness theorem and in Section ^ compute the topological obstruction to pose a Fredholm 
boundary value problem for an elliptic operator. Examples of boundary value problems 
for the Hirzebruch operator are presented. 

We are grateful to Professor G. Rozenblioum of Chalmers University, Goteborg, Swe- 
den for numerous helpful discussions and Dr. V. Nazaikinskii for help on a number of 
topics. The results were announced at the Conference "Workshop in Partial Differential 
Equations", Potsdam, Germany, November 12-16, 2001. We would like to thank the 
organizers of this conference for their hospitality and support. The work was partially 
supported by RFBF grants 00-01-00161, 01-01-06013, 99-01-01254. The paper was par- 
tially written at Chalmers University of Technology and supported by a grant from the 
Swedish Royal Academy of Sciences. 

1 Symbolic algebra 

Let TT : M — > X be a locally trivial fiber bundle of compact smooth manifolds M and 
X with a typical fiber Y. Manifolds M, Y, X are assumed to be closed. The fiber tt~^{x) 
over x e X is denoted by Y^. 

On the total space of the fibration, we consider special coordinates described as follows. 
For a domain U G X with coordinates Xi, . . . , x„, n = dimX and a trivializing mapping 
au 

^-i(f/) ^ UxY 

u 



2 



we consider some coordinates yi, ■ ■ ■ ,ym, ^ = diniF in a domain V d Y oi the fiber. 
Denote the dual coordinates in the cotangent space by (^i, ... , 771, . . . , 77^)- 

Definition 1. Principal symbol of order zero is a function 

om e C'^{T*M\tx*T*X), 

which is homogeneous in covariables of order zero and is directionally smooth as covari- 
ables approach the plane tt*T*X C T*M. More precisely, in local coordinates we suppose 
that the function 

a{x, y, e, tr/) , (x, y,C,t,v) E U x V x (M" \ {0}) x M+ x (R™ \ {0}) (1) 
extends smoothly up to t = 0. 

The following notation will be used for the directional limit 

auix, y, ^, T]) = Jim omIx, y, ^, tr]) 

of the principal symbol at the plane 'k*T*X. This function is defined if both |^| 7^ and 
\ri\ 7^ 0. It is homogeneous of order zero with respect to ^ and 77. 

It is clear that the restriction of the principal symbol to the cosphere bundle S*M can 
have discontinuities. However, our definition is equivalent to the requirement that the 
restriction of the principal symbol to S*M extends to a smooth function on a compact 
manifold S*M \ tt*S*X with boundary. This space is a compactification of S*M \ ti*S*X 
obtained by attaching to it sequences {xi,yi,^i,rii) converging to a point in Tr*^*^ such 
that the quotients ?7/|?7| converge as well. One can show that this manifold is diffeomorphic 
to S*M \ Ut,*s*x, where Ut,»s*x is an open tubular neighborhood of Tr*^*^. 

Definition 2. Operator symbol of order zero is a function 

ax e C~(T*X\{0},^°(y)) 

that at a point (x, ^) G T*X \ is a classical pseudodifferential operator of order zero in 
the fiber Y^. The function ax is assumed to be homogeneous in covariable of order zero 
and smoothly depending on (x,^). More precisely, ax(a^,0 is a smooth function with 
values in a Frechet space. 

Remark 1. Let us recall the definition of the Frechet structure on the space of classical 
pseudodifferential operators, see P, The set of seminorms is defined as follows. Fix on 
Y a quantization mapping a t-^ a, defined for smooth homogeneous symbols a. Then a 
classical ipDO A of order d on F is equivalent to a sum 

A-^a,_,-, aa-jeC^{S*Y). 
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Here the symbols a^, are homogeneous of degree k. While the equivalence means that 

Af>j>0 

In these terms, the following two systems of seminorms on the space \E''^(y): 

1) ||v4||qj = ||arf_j||a, where a runs over a countable set of seminorms on C°°{S*Y); 

<N-i '^d-j\\H''iY)^H'>-('i-'^'>iY)- This is an operator norm; 
define the Frechet structure. 

Definition 3. Symbols and ax are said to be compatible if 

a{ax)=aM, (2) 

i.e. the principal symbol of the operator symbol ax is equal to the limiting value Hm of 
the principal symbol at the horizontal bundle tt*T*X. 

Definition 4. By S(A<f, tt) denote the algebra of compatible pairs {aM,ax) with a com- 
ponentwise product. An element a G S(M, vr) is called a symbol on M. 

Example 1. Let B G C°°{X, \E''^(y)) be a smooth family of pseudodifferential operators 
of order zero in the fibers: 

B, : C^iY,) 

Then the pair 

is a symbol on M. Indeed, the principal symbol a{B){x,y,r]) is smooth for \ri\ ^ 0, while 
the rescaling as in (|I]) does not change the symbol: 

(t{B){x, y, tr]) = a{B){x, y, r/), 

since cr(i?) is homogeneous of order zero. Thus, in this case cr(i?) = cr{B) and the 
compatibility condition (|^) is clearly satisfied. 

Example 2. Let us now choose a smooth symbol a G C°°{T*M \ {0}). Then the pair 

(a, oItt.t'x), 

where operator symbol a|^,^^^ = a(x, y, ^, 0) acts as a multiplication operator, is a symbol 
in the sense of Definition |[ 

Denote by Sq C S(M, tt) the subalgebra multiplicatively generated by the symbols 
from Examples 1 and 2. 
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Proposition 1. Eq is dense in S(M, vr), where E(M, vr) is equipped with the Frechet 
structure as a subspace: 

S(M, tt) c C^{S*M \ U^.s'x) © 

Proof. 1) Let us show that the subalgebra of principal symbols corresponding to Sq is 
dense. 

Consider a principal symbol a{x,y,^,ri). By a partition of unity argument, it suffices 
to approximate Q a over a neighborhood of an arbitrary point {xq, Uq) G M, where we can 
use local coordinates. In the domain |?7| > 2e\J^'^ + rf, where a is smooth, it is equal to 
a smooth symbol 

where the chopping function is zero for t < e and is 1 for t > 2e. Thus, the main 
problem is to approximate the symbol for small r]. 

To this end, consider the function a{x,y,C,',tri'). It is smooth for 

l^'l = \rj'\ = 1 and t G [0,100]. (3) 

Hence, it can be approximated by a sum of products of functions depending on two subsets 
of variables and {y,0- 

N 

(^{x,y,^',tr]') ^^aj{x,^',t)bj{y,ri'), 

where the terms are smooth functions for the parameter values as in (^. Thus, the 
original symbol is approximated for small t] by the expression 




for |?7| < 100|^|. Here the terms bj have the desired form as in Example 1 and they belong 
to the subalgebra Eq. However, the functions aj are not smooth at rj = (they contain 
terms \ri\). Let us eliminate this singularity. 

To this end, consider the Taylor expansion of aj{x,^',t) in t at zero. We obtain 

^Here and in what follows we use the term approximation in the sense of Frechet topologies on the 
corresponding spaces. 
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For k even the corresponding term is smooth for |^| 7^ 0. However, for k odd the 

quotient is not smooth. We represent it in the form 

where <fi denotes a partition of unity on the sphere |?7| = 1 such that rji does not vanish on 
the support of ipi. In this sum, the factors on the left correspond to symbols from Example 
1, while the quotient fJi/lCl is smooth and corresponds to a symbol from Example 2. 

Now the approximation of the symbol a is defined as follows. First, we approximate 
a for \ri\ < \^\ by the formula (H). Then the Taylor expansion in (|^) is written with the 
decomposition of nonsmooth factors as in (^. The error term in the Taylor expansion 
shows that for Ir^l < 4£:|^| (for e sufficiently small), the approximation is good. Denote 
the corresponding approximating element by 

a 

(tta and ba are homogeneous in (^,77) and rj, respectively). Then the expression 



approximates the initial principal symbol with a small error as desired. This approxima- 
tion can now be globalized in x and y hy a partition of unity. 

2) Let us show that a compatible pair {pm,Px) of a principal symbol and an operator 
symbol can be approximated by elements from the subalgebra Sq. 

It follows from the previous part of the proof that we need to prove this for the trivial 
principal symbol Pm = only. 

So, let us consider a symbol (0,px) defined by an operator- valued function Px{x,^) G 

Similar to the above, let us approximate px in a neighborhood of a point (xo,^o) ^ 
S*X. There px can be approximated by its Taylor expansion in x and ^. We write this 
for brevity as 

px{x,o= Yl pj,.o,u^,OBj+oi\x-xo\'' 

where Bj are pseudo differential operators in the fibers and Pj,xo,ioi^yO smooth (scalar) 
functions. 

Taking a cover of S*M with sufficiently small charts Ua and using the corresponding 
partition of unity ipa, we obtain an approximation 



Y^aPj,x^,iAx,^)Bj 
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for px- If the charts C/q, are chosen good enough (e.g., their diameters are less than e 
and every point belongs to at most / + 1 charts, where / is the dimension of S*X) then 
this expression is close to px in the Frechet space S(M, tt). To end the proof, it suffices 
to show that a term of the form (pp{x, is equal to a composition of a smooth symbol 
and a symbol of a family of pseudodifferential operators. Indeed, take x ^ C°°{X) such 
that 7r*x = 1 in a neighborhood of the support of </?. Then 

ipp{x,^)B = [ipp{x,^)]xB. 

Here xB is a family of ipDO in the fibers, while (pp{x, ^) is the value for 77 = of the 
smooth symbol 

where x{t) is zero for \t\ > 2e and 1 for \t\ < e. This completes the proof of the proposition. 

□ 



2 Operators on fibered manifolds 

The aim of this section is to show that a symbol a e E(M, tt) defines an operator 

a : C°°(M) — > C^{M) 

acting on C°° functions on M. Let us construct it. For a decomposition a — (om, dx) 
this operator is defined as 

au^ {clm + ax -aM)u, ueC°°{M), (7) 

where the first component qm denotes the usual quantization of the principal symbol, 
while the second corresponds to quantization of operator-valued symbols (note that there 
is a correction term Tim to the operator symbol. This will be described later on). 
More precisely, gm is first defined over charts Ua <Z X by the formula 

^-^ = Y1-(^^^M J e'^^^+y^^aM(x,y,^,rj)(^)i^,rj)d^drj, (8) 

where Xj iv) is a partition of unity subordinate to an atlas in the fiber, ipj is equal to 1 
in a neighborhood of the support of Xj and vanishes far from it. Finally, the hat denotes 
the Fourier transform in x and y. 

The operator symbol qm is defined along the same lines 

'^m{x,Ov = -^^^^^i^jiy) j e'y'^aM{x,y,^,r]){vXj){v)dri, v e C°°{Y^), 
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quantizing only in the fiberwise variables. This definition uses the same data Xjii^j 

In a chart Ua C X, the operator ax — o-m with operator-valued symbol ax — o-m is 
defined as 

B-^ = J^. j e'^^[ax{x,0-^M{x,i)\mdi. (9) 



TiX 



Here u(^) denotes the Fourier transform in x of u{x, y). 

Globally on M the operator a is defined using the local expressions Aa, Ba- 

a 

by a partition of unity x'a subordinate to the atlas {Ua} and functions ip'^ supported 
in Ua with the property i'aX'a — x'a- 

Remark 2. If a = (a,a|7r*T*x) corresponds to a smooth symbol as in Example 2, then 
a is just the usual ipDO on M with symbol a. On the other hand, for a family B as in 
Example 1 consider the symbol cr = [cr{B), B). Then one can prove that a = B. 

The quantization formula (0) resembles the formula for operators with discontinuous 
symbols on the circle, see 0. 

Theorem 1. a extends to a hounded operator in Sobolev spaces H^{M), s G M 

a : H'{M) H\M). 

Proof. 1) The continuity in H^{M) of the first component (P) corresponding to the prin- 
cipal symbol was proved already in 0. It was shown that no regularity of the symbol in 
covariables is required to obtain continuity in Sobolev spaces (for all s). Moreover, the 
norm of an operator with symbol a(x',^') in H'^{M) is bounded by the maximum of a 
finite number of derivatives in the geometric variables: 



max sup \d^,a{x' ,^')\, (11) 

up to a constant depending only on s (the order N of the derivatives also depends on s). 

2) The continuity of operator (|^) with operator-valued symbol in follows from the 
paper In this case the L^-norm is also bounded by estimates of the form ([TI|) , where a 
is replaced by the corresponding operator-valued symbol and the absolute value is replaces 
by the operator norm. 

3) Let us prove that the term (||) defines a bounded operator in the Sobolev spaces. 
By interpolation it suffices to prove the boundedness for s = ±2m, m G N. 
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If s = 2m then according to Theorem ^ of the Appendix an operator 

qx : C°°{M) C^{M) 
with operator-valued symbol qx{x,^) E \I'^^(Kc) 

Qx = 0.x — Om 
acts continuously in H'^"^{M) if the composition 

iAY + erqxix,Oi/^Y + e)-"' ■■ C^iY)^C°^{Y) (12) 

is uniformly bounded in L'^{Y), where Ay is a positive Laplacian on Y. 
The corresponding estimate 

ue + Ayrqx{x,oie + /^Y)-n\LHY)^mY) < c 

can be proved using a decomposition of the commutator 

-2<j<2m-2 

where the component aj has order j and its norm in the spaces 

ajix,0--H'iY)^H-'-\Y) 

is bounded by (1 + |^|)^™'~^~-'. This decomposition can be proved by induction. 
Using this decomposition, we can estimate each of the terms: 

\\Oj{x,^){C,'^ + Ay)~'^\\l2(Y)-*L'2{Y) < \\Oj\\H3{Y)^H0{Y) " \\{^Y + (.'^)~"^\\HO{Y)-^Hj (Y)- 

Thus, we obtain 

\\a,{x,Oie + Ay)-|U.(y)_^.(y) < (1 + ■ max ((1 + 1^1)-^"^+^ (1 + lel)"'™) . 

This expression is clearly uniformly bounded. 

The remaining case s = —2m can be considered similarly. 

□ 

3 Operator algebra 

Theorem 2. The following composition formula is valid 

aia2 = d\(T2 + K, 

where ai,a2 E S(M, vr) are symbols, while the error term K is a compact operator in all 
spaces H^{M). Moreover, if ai E Sq then K has order —1 in the scale. 
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The proof of this theorem is done in the Appendix. 

Definition 5. Denote by \E'*'(M, vr) the space of operators of the form 

a + K : C°°(M) — ^ C°°(M), 

where operator K extends to a compact operator in the scale H^{M). 
The composition formula enables us to prove the following result. 

Theorem 3. ^°(M, vr) is an algebra. 

2) The subalgebra \l/o C \E'°(M, vr) generated by ijjDO's on M, families of ipDO's in 
the fibers and compact operators is dense in \I/°(M, vr) with respect to operator norm in 
H'{M). 

3) The norm, modulo compact operators, is equal to 

inf ||a + = maxi sup |aAf(x, y, ^, r^) I, max ||ax(a;, Oil ) , (14) 

k&K{L^(M)) \{x,y,^,rj)eS*M\7T*S*X (a^,C)6S*X J 

where K{L'^{M)) denotes the ideal of compact operators. 

Proof. 1 ) This straightforwardly follows from Theorem ^. 

2) Suppose that a G \I'°(M, vr). According to Proposition [I| its symbol cr can be 
approximated by a symbol cTg G Sq. Then by Theorem ^ and Remark 2 we have G \l/o. 
On the other hand, the difference a — as = a — as has a small symbol. Thus, its norm is 
small. This proves that \l/o is dense in \1/°(M, vr). 

3) By virtue of the second part of the theorem, it suffices to prove the estimate for 
a G \I^o and a = {aM, ax)- 

Let us first prove the estimate from below. Suppose that (xq, yo, C,o, rjo) G T* M\k*T* X . 
Then we choose a sequence of smooth functions 

w„(x,i/,t) = e^*(^'«°+^*)Xn(x,i/), 

where ||Xn||L2(M) = 1 and Xn is supported in a ball of radius 1/n around (xo,?/o)- 
Since a G \l/o, it has the form (modulo a compact operator) 

^ = ^X\AapB^p, (15) 

a (3 

where A^is are ipDO^s on M and Bajs are families of ipDO^s in the fibers as in Example 1. 
Then as t tends to infinity we obtain 

AapUn = a{Aaf3) {x, y, ^q, ?7o)m„ + O 
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according to the Hormander's definition of pseudodifferential operators. 
Hence, as t ^ oo we obtain 



aun = aM{x, y, ^o, '7o)Mn + O 



On tlie otlier fiand, Un weakly converges to zero as t ^ oo. Tlius, for a compact operator 
K we have Kun —>■ 0. Therefore, 

(a + K)un = aM^Xo, yo, ^q, r]o)un + Ue + O + KUn, 

where Ue = {aM{x,y,$,,ri) — aM{xo,yo,$,o,r]o))un, and for n large we have \\us\\ < e. For t 
large this yields 

\\(a + K)un\\L^M) > \0'M{xo,yo,C.o,r]o) \ ■ Wunh^M) - £■ 

Since e can be chosen arbitrarily small and {xo,yo,C,o,Vo) and K are also arbitrary, we 
have the desired estimate of the norm modulo compact operators 

mi\\d + K\\> sup \aM{x,y,^,r])\. 

^ (x,y,^,r])€T*M\-K*T*X 

Let US prove the second part of the lower estimate that deals with the operator symbol. 
For (xo,^o) £ S*X and v G C°°(Yx^^) consider the sequence 

Un{x,y) = e'''^'''v{y)xn{x), 

where Xn{x), \\Xn\\L'^(x) = 1 is a function on the base X supported in a ball of radius 1/n 
around Xq. 

For a as in (|15|), we obtain 

Aa/SUn = (t{A) (x, y, ^0, 0)Un + O ^ 

and 

n 

By an argument similar to the previous part of the proof, we obtain 

inf ||a + A;||> max Waxix.ClWh'^iY \^l'^(y \- 

11 



We now prove the estimate from above. This is done following the standard scheme 
(see, e.g. [g). 

Let us denote by ||cr|| the norm of the symbol a = {aM,(ix), as defined by the right 
hand side of (p^. Fix a constant C > \\a\\. Then — a*a is self-adjoint and positive 
symbol. Denote by ao its positive square root 

G — a a = (Tq. 

Let us now approximate ao by a symbol ae G So- Thus, according to the composition 
formula, we have for the corresponding operators 

where K_i is a compact operator of order —1 in the Sobolev scale and has norm less 
than e. This formula gives the following estimate 

||5''u|||2(M) < C'1^IIl2(m) + \\K-iu\\l2{m) ■ ll^^l|L2(Af) + £\\u\\h(M)- (16) 
Denote by the smoothing operator on L'^{M) with symbol 



a{Re)ix,y,^,ri) 



0, for + Ir^p > 2/e, 

1, for + Ir^p < 



It is a compact operator and the following estimates are valid (see [^) 

\\u- R^u\\l2(^m) <\\u\\l^m) and \\u ~ Reu\\H-i{M) < c{e)\\u\\L2{M), (17) 
where c{e) ^ as £ ^ 0. Then (|l^) and (|T7|) give the desired estimate: 

\\a{u - ReU)\\l2(^M) < C^hWhi^M) + C(^)lklli2(M) 

with C(£:) — >■ as £ — 0. Hence, if we take the compact operator k = aRe-, this yields for 
the infinum of the norm the following estimate 

inf IIct + fcll < 

k&K{L'^{M)) 

as desired. 

Theorem is proved. 

□ 

Denote by \1/''(M, vr) the closure of \I/''(M, vr) with regard for the operator norm in 
L?'{M). The next result describes the corresponding Calkin algebra. 
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Corollary 1. 

^°(M,7r)//C ~ S(M,7r), 

where K, denotes the ideal of compact operators and E(M, tt) is the completion of the 
symbolic algebra S(M, vr) with respect to the norm || ■ ||. In more detail, the completion is 
a suhalgebra 

S(M, tt) c C{S*M \ U^^s^x) ® 
consisting of compatible pairs {aM, ox)-' 

S(M,7r) =[aM®ax\ o.m\qu^,^,^ =cr{ax)}, 

where Ut,*s*x denotes an open tubular neighborhood of t:*S*X in S*M and "^^{Y) is the 
norm closure of the algebra of pseudodifferential operators of order zero in the fibers. 

The proof of the corollary follows easily from the estimate modulo compact operators. 
The description of the Calkin algebra enables us to state the Fredholm criterion. 

Corollary 2. G ^°(M, vr) is a Fredholm operator in Sobolev spaces if and only if a is 
invertible. 

Proof. The "if" part follows from the composition formula: the parametrix is given by 
a~^. The "only if" part is proved as follows. Let us assume at first that s = 0. Suppose 
that a is a Fredholm operator in L^(M) with a left quasiinverse A: 

Aa = l + Ki. 

Then this gives an apriori estimate 

\\u\\ < C\\au\\ + \\Kiu\\. 

If we substitute in this inequality the sequence Un{t), as in the proof of Theorem we 
obtain, choosing n big enough and letting t — > oo, that 

1 < Ci\aM{xo,yo,^o,r]o)\ 

and a similar estimate for the operator symbol 

l<C2||(Tx(xo,eo)|| 

(these estimates can be obtained choosing approximations of a by elements of the subal- 
gebra \E'o). Thus, the symbol is monomorphic. Passing to the adjoint operator, one proves 
the surjectivity. 

Therefore, the symbol of a Fredholm operator is an isomorphism. The ellipticity 
of Fredholm operators in if*(M), s ^ can be proved along the same lines using the 
compositions A^'^^aA^/''^. □ 
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Remark 3. A generalization of the composition formula and the finiteness theorem to 
operators acting in the sections of some vector bundles over M is rather standard and is 
left to the reader, e.g. see 0. 

4 Elliptic Theory in subspaces 

Let us consider operators 

D : ImPi — > ImPs (18) 

acting in subspaces defined by projections 

Pi : C^{M,E) — > C^{M,E), Ps : C^{M,F) — > C°°{M,F), 

where E, F are vector bundles over M. We suppose that Pi 2 belong to the algebra defined 
in previous sections: Pi 2 € \1'°(M, tt) and operator D is a restriction of some operator D 

D : C°°{M,E) — >C°^{M,F), 

also from our algebra: D G \E'°(M, vr). 

Theorem 4. Operator ([T8|) defines a Fredholm operator in Sobolev spaces if and only if 
the following two conditions are satisfied: 

1 ) the principal symbol 

aM{D) : ImcrAf(Pi) Im(TM(-P2) 

is invertible over S*M \ n*S*X; 

2) the operator symbol 

ax{D) : Imax(Pi) ^Im(Tx(P2) 

is invertible over S*X. 

Remark 4. In more detail, these conditions require that the principal symbol is a vector 
bundle isomorphism and the operator symbol is an invertible family of operators in sub- 
spaces defined by pseudodifferential projections (see, e.g., 0). Let us also mention that 
the symbol of a projection is a projection itself. 

Proof. If the two symbols are invertible then the parametrix has the form: 

D-^ = PiPi : ImP2 — ^ ImPi, 



14 



where a = {a^j{D),o'x{D)) is the symbol of D. The fact that the differences D^^D — 1 
and DD~^ — 1 are compact operators follows from the composition formula. 

Let us prove the "only if" part. If D is a Fredholm operator, then the direct sum 

Im Pi © Im ^-^^ Im Pi © Im 

also has the Fredholm property. Moreover, there is an obvious isomorphism 

Im Pi © Im P^ c^H\M,E). 

Hence, D*D © 1 is an elliptic operator and its symbol has trivial kernel. The surjectivity 
of the symbol is proved along the same lines using the composition DD*. 

□ 



Remark 5. Consider a subclass of elliptic operators in subspaces (0), where both pro- 
jections Pi, P2 correspond to families of pseudodifferential projections in the fibers. Then 
for the identity map tt = id : M — ^ M we obtain the usual operators on vector bundle 
sections over M, while for the collapsing map tt : M —>■ pt this construction gives the 
class of operators acting in subspaces defined by pseudodifferential projections (see 0). 



5 Boundary value problems on manifolds with fibered 
boundary 

1. Main definitions. Consider M a compact smooth manifold with boundary 
denoted by dM. Assume that dM is the total space of a locally trivial fiber bundle 



71 : dM 



X 



with a compact base X and a compact fiber Y as in the previous sections. 
On M we consider an elliptic differential operator 

D : C^{M,E) — > C^{M,F) 

of order d. To define the boundary conditions for D, we introduce the operator 



J : ^^{M^E) — > C°°(M,E| 



dM> 



that maps a function to its jet of order d in the normal direction to the boundary: 



ju 



d 



u\ 



dM 



.d_ 

''dt 



u 



dM 
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Here t > denotes the normal coordinate near the boundary. We consider boundary 
value problems of the following form: 

'Du = f, ueC°-{M,E), feC°-{M,F), 

Bju = gelmP, ImP C C°°{dM, G), G G Vect {dM), ^ ' 

where the subspace ImP in the space of sections of a vector bundle G is defined by a 
family of pseudodifferential projections over X acting on functions in the fibers of vr. 

If D is a first-order operator, then the boundary condition B is assumed to be an 
element of the algebra \l/°(c}M, vr). For operators of higher order the boundary condition 
is more complicated. In this case it has d components 

B : C^{dM,E\l^) ^C°°{dM,G) 
and is defined composition 



B — {Bq, PiA^I/^, . . . , Prf_i Ag^/ ^ 



where A^Af is a positive Laplacian on dM and the components Bj belong to \I'o(9M, vr). 

Remark 6. This construction reduces to some well-known classes of boundary value 
problems for special types of projections vr: 

1) vr = id : dM dM. In this case we obtain classical boundary value problems; 

2) TT : dM — > pt. This gives boundary value problems in subspaces (see [pi]|]); 

3) vr : dM — > X and vr is a covering. This gives a class of nonlocal boundary value 
problems studied in 0. 

2. Finiteness theorem. The ellipticity condition of a boundary value problem for 
D is formulated in terms of a special vector bundle 

L+{D) G Vect{S*dM) 

defined as a subbundle in the pull-back of -f^jg^j to S*dM and generated by the Cauchy 
data at t = of functions u{t) satisfying the ordinary differential equation 

aiD) (^x', 0, e', uit) = 0, ix', a e S*dM, 

that remain bounded as t +oo (see By L^(D) let us denote some subspace in 

C°°{dM, E^gj^^) that is defined by a pseudodifferential projection Q on dM with sym- 
bol projecting on the subbundle L^{D) C -f^j^j^^ along the complementary subbundle 
L_{D) corresponding to solutions decreasing at — oo. Projections Q are called Calderon 
projections for D. 



The following Fredholm criterion is valid for boundary value problems (|T^) 
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Theorem 5. Boundary value problem (^) defines a Fredholm operator for s > d/2 

PH'-^/\dM) 

if and only if two conditions are satisfied: 

1) the principal symbol of B is invertible on S*dM \ 'k*S*X 

(Tom{B) : ImaoMiQ) — >lma{P); 

2) the operator symbol of B is invertible on S*X: 

ax{B) : Im(Tx(g) — ^ ImP. 

Here Q is a Calderon projection for D and for d > 1 we denote by aQM{B) and ax{B) 
the symbols of the tuple {Bq, . . . , Bd-i). 

Remark 7. It should be noted that the conditions of the theorem use only the principal 
symbol cr{D), symbol of B and the projection P. 



Proof. By the results of [10|, the boundary value problem has the Fredholm property if 



and only if an operator on the boundary 

(5o,5i,... ,5d„i)|i^Q : Img-.ImP 

has the Fredholm property. We apply the Fredholm criterion stated in Theorem ^ to this 
operator in subspaces. 

This readily shows that conditions 1) and 2) of the present theorem are necessary and 
sufficient for the Fredholm property to be valid. 

□ 

Example 3. (Elliptic boundary value problem for the Hirzebruch operator). Let M"^^ 
be an oriented 4/c-dimensional manifold with boundary. Suppose that the boundary is a 
product 

dM = X'"" X Y"'^'^. 

In a neighborhood of the boundary choose a metric corresponding to the Cartesian product 
[0, 1) X X X Y. Consider the Hirzebruch operator 

DM = d + 6 : A+(M) — > A-{M) 
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on M, where d and 5 are the exterior derivative and its adjoint and A^(M) are the spaces 



of dual and antiselfdual forms on M (e. g., see [|T2[). Then near the boundary Dm can be 
represented in the form 



d_ ( Dy D 

dt ^ 



X 

Dx -Dy 



where Dx denotes the Hirzebruch operator on X 

Dx : A+(X) ^A-(X) 

and Dy denotes the odd analog of the Hirzebruch operator on Y (see [T^) 

Dy : A*{Y) — > A*(F), 



(20) 



r 



id. 



Dy = r{dy + t\^^^^^= 

It is elliptic and self-adjoint. Denote by 11+ 

n+ : N*(Y) — > N*(Y) 

the nonnegative spectral projection of Dy and n_ denote the complementary projection. 
Consider the boundary value problem 



d_ 

dt' 



dM 



dM 



Dy D\ 

Dx -Dy 
= (7i G Im n 
= (72 G Im n 



(21) 



Proposition 2. Boundary value problem ( |2T| ) has the Fredholm property. 

Proof. 1) According to Theorem]^, it suffices to check the invertibility of the corresponding 
principal and operator symbols. 

2) An elementary computation shows that the principal symbol of the Calderon pro- 
jection Q on S*dM is the matrix 

^aM{Q)--[^ a(Dx) l-aiDy] 
Hence, the operator symbol of the Calderon projection is 

= 2 Udx) 1 
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We need to prove that the two maps 

ImaoMiQ) ''^^ Im(7r+ © 7r_), 
ImaxiQ) Imn+ © ImH^ 

are isomorphisms (here 7t± denote the principal symbols of II-i-). To prove that the two 
maps are isomorphisms one can compute their compositions with the maps in the opposite 
direction 

Im(7r+ © 7r_) "^-^^ ImaaAiiQ), 
Imn+ © Imn_ ImaxiQ). 
An explicit computation shows that the compositions 

(7r+ ©7r_)(TaA/(g), aaM(Q)(7r+©7r_), (n+ © n_)orx(Q), ax((5)(vr+ © 7r_) 

have trivial kernels. Thus, the boundary value problem (|2T|) satisfies the assumptions of 
Theorem |^ and, consequently, has the Fredholm property. 

□ 

6 Topological obstruction 

There is an obstruction to define a Fredholm boundary value problem for a given 
elliptic operator D on M. 

Theorem 6. Suppose that an elliptic differential operator D on M has a Fredholm bound- 
ary value problem of the form (]TP|). Then the principal symbol (j{D) at the boundary has 
the following property 

7r!KD)|aM] = 0, (22) 

where 

[(j{D)\qm] e K{T*dM X M) ~ K\T*dM) 
is the difference construction and 

TT, : K\T*dM) K\T*X) 
is the direct image mapping in K -theory under the projection vr. 
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Proof. 1) Let {D,B,P) be an elliptic boundary value problem for D. This means that 
there is a Fredholm operator 

(So,Ei,... : ImQ^ImP, (23) 

where Q denotes a Calderon projection as previously. 

The existence of a Fredholm isomorphism (|23|) implies that the symbols of the two 
subspaces Im Q and Im P are homotopic. 

More precisely, consider both Q and P as projections in the direct sum 

C^{dM, E\'Ij © C^idM, G). 
Then the homotopy of the principal symbols is defined as 

QdM,^ = o-{Q) cos^ (p + a{P) sin^ (p + 2a{P)aaM{B)o{Q) sin*/? cosy? (24) 
and a similar formula is valid for the operator symbol 

(lx,f = crxiQ) cos^ (p + Psin^ (f + 2Pox{B)ax{Q) sin cos v^. (25) 
This is a homotopy of compatible symbols and we obviously have at = 

qdM,o = cr{Q), qxfl = CTxiQ) 

and for = 7r/2 

2) On the other hand, let us represent the topological invariant in ( ^21) in analytic 
terms. It is well known that the element [(T(D)|gM] ^ K^{T*dM) can be expressed in 
terms of the symbol of the Calderon projection for D: 

[a(D)|aM]= [{2a{Q){x,y,^,r])-l)^e + v' + ^r] eK{T*dMxR), (26) 

where the coordinates correspond to the fibration tt : dM — ^ X and r G M denotes an 
additional variable. The element is understood in the sense of the difference construction, 
since it is an isomorphism outside a compact set. 

Let us represent the element in Eq. (^) as the difference construction in the sense of 
for a family of elliptic operators in the fibers, where the parameter space is the product 
T*X X M. More precisely, we will define a family of elliptic operators parametrized by 
B{T*X X M), where B denotes the unit ball bundle. The family will turn out to be 
invertible over the spherical bundle S{T*X x R). Therefore, its analytic index is in the 
following relative group 

K{B{T*X X R), S{T*X x R)) ~ K{T*X x R). 
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The desired family of elliptic operators denoted by D{x,^,t) for the parameters 
(x, ^, r) G B{T*X X M) corresponds to a family of symbols 



2a(Q) (a;, ^ (l " - ^')) " l] (l^l + 1 - + r^)) + 



IT. 



(27) 



The invertibility of this symbol for + < 1 can be verified by an explicit computa- 
tion. It is clear that on the unit spheres for + = 1 this symbol is a vector bundle 
isomorphism independent of 77: 

{2a{Q)ix,y,^,0)-l)\^\+tT. 

Hence, the difference construction for the family of elliptic symbols (pTj) is an element 
of the group K{T*dM x R). 

One can show that this difference construction for the family D{x,^, r) coincides with 
element (^61) . Hence, by the Atiyah-Singer formula for families we obtain the desired 
expression in analytic terms: 



TTi 



,HD)|^J =ind [D{.,.,.)]eK\T*X), 



where 



2a{Q)(x,y,^,^^{l-e-r')]-l 



IT 



{2i 



(29) 



and the hat means that we have a family of operators in the fibers. 

We will show that this family of elliptic operators is homotopic to an invertible family. 
Therefore, the index is zero in this case. 

Denote by Q{^, t) the following family of pseudodifferential operators in the fibers 



On the sphere + = 1 for ^ 7^ this is a family of projections, while for other values 
of the parameters (5(^, r) is only an almost-projection. One can verify that this property 
implies that (|2|) is elliptic on B{T*X x M) and invertible on ^(r*^ x M). Let us define 
the homotopy D^{x, ^, r) by changing this family Q(^, r) only: 



D^{x, e, r) = (2Q^(e, r) - 1) del + 1 - (^^ + r')) + zt, 



(30) 



such that Qip{^,T) satisfies the above mentioned property. The homotopy Q,p{^,T), (f G 
[0,7r/2] is defined on the spheres as 



Qv^(e,r) = gx,^(||), for e + r' = l, 
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(the family gx,(^ was defined in (P^), while inside the balls for + < 1 it suffices to 
define a homotopy of the corresponding symbols: 

(the principal symbols qaM,^p were defined in (|2^)). For brevity we omit the geometric 
variables x, y in the formulas. 

One can verify that for this choice of the operator in (^) is elliptic for + < 1 
and is an isomorphism for + = 1. 

At the end of the homotopy for = 7r/2 the operator Q^{^, r) is a pseudo differential 
projection that does not depend on and r. Hence, for ip = 7i/2 the family D^{x, ^, r) is 
invertible on the entire space B(T*X x M). Together with (^) this yields the desired: 

^HD)\^m\ = [mdZ^|(-,-,-)] =0. 

□ 

Example 4. Similar to Example 3, suppose that dM'^^ = X""^ x Y'^'" and consider the 
projection 

j^odd ^ y^et) ^ ^ j^odd 

with an even-dimensional fiber Y^^ . 

Now the Hirzebruch operator Dm acquires the form 

d_ (Dx D*y 
dt \Dy -Dx^ 

In contrast with the previous Example 3, the obstruction does not vanish in this case. 
Lemma 1. 

7r,[a(D)|gJ = mdDY[a{Dx)] G K\T*X). 
Proof. This follows from the index formula for families. 

□ 

According to this Lemma, the Hirzebruch operator has no Fredholm boundary value 
problems if indDy 7^ 0. When rndDy = 0, consider a boundary value problem 

'u\ f fi 



d_ (Dx D*y 

dt^KDy -Dx 



' (31) 



Proposition 3. Boundary value problem (31) has the Fredholm property. 

Proof. The check of the ellipticity of the boundary condition is similar to the one in 
Example 3 and is left to the reader. 

□ 
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7 Appendix 



In this appendix we prove two important technical results. First, we establish the 
boundedness in the Sobolev spaces for a class of operators with operator-valued symbols. 
Second, we prove the composition formula from Section ^. 

Theorem 7. Suppose that an operator-valued symbol p{x,C,) G \E''* (F) of order d is de- 
fined for X G M", vanishes outside a compact set and satisfies the estimates 

;i + A,)- (A, + et^'^'^H^^O {Ay + < (32) 

uniformly in x and ^ for some N > {n + \s — d\) /2. Then the operator 

P : (M" xY)^C^ (M" X Y) 

with operator-valued symbol p extends to an operator of order d in the Sobolev spaces and 
an estimate of its norm is valid 

ll-^ll_ff^(R"xy)^Hs-''(R"xy) ^ C'p ■ C (s, d) , 
where the constant C (s, d) does not depend on the operator. 

Proof. Let us estimate the norm \\Pu\\^s-diTa,n^v\ ■ In terms of the Fourier transform in x 
this norm can be represented as 



I Pii II 

I-* "ll_ffs-d(]Knxy) 



2\{s-d)/2 



L2(IR",L2(y)) 



Since 



we obtain that 



Pu= I e'^^p{x,0u{0d^, 



Pn{0 = j p{C-^,0n{0d^, 

where p denotes the Fourier transform in x of the operator symbol. Using this expression, 
we obtain 

2 



I PiiW 

l-T "lli^s-d(IRnxy) 



L2(K",L2(y)) 



This shows that \\Pu\\jjs-d(^^ny^Y) equal to 



(Ay + c^) ^'"'^^'pic - (Ay + e) (Ay + e) u (0 di 



(33) 
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Let us now estimate the norm of the product in this formula. Consider first the term 



A I a2 \ {s—d)/2 



L^{Y)->L'2{Y) 



The estimate (|3^) of the theorem imphes that 

.2\ {s~d)/2 



A{^, C) < Const 



Ay + C 

Ay +e 



■ Cr) 



1 



L'^{Y)^L\Y) (l + IC - ^1 ) 

On the other hand, the first term can be estimated by a Peetre type inequahty 

.2\ {s-d)/2 



Ay +e. 

' L'^Y)^L^{Y) 

Thus, the term y4(^, () is estimated as 

A{^, C) < Const ■ Cp 



< Const {l + \C-^ff~'^^' 



(1 + ic-er) 



2^N-\s-d\/2- 



We are now in a position to estimate 



\Pu\\jjs-d(Jgi^ny.Y) — 



J Ai^,c)\\{^Y+ef'uio 



L\Y) 



L2(]R") 



This gives 



\Pu\ 



H'>-d{K^xY) 



< Const ■ C„\\u 



dC 



where, for > (n + \s — d\) /2 the last integral converges. This proves the boundedness 
of P and the corresponding norm estimate. 

□ 

Proof of the composition formula. The rest of the appendix contains the proof 
of the composition formula 

ai(T2 = 0102 

for two symbols cri,cr2 G S(M, vr). The comparison is valid modulo a compact operator 
in the scale of Sobolev spaces. 
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Lemma 2. If the composition formula is valid for all ai e Eq, (72 G E(M, tt). T/ien is 
true in general. 

Proof. Consider a pair of symbols Ui, (T2 G E(M, tt). Choose an approximation of ai by an 
element of the dense subalgebra Eq C S(Af, tt) (see Proposition 1). Then the difference 
of the corresponding operators is denoted by 

and has norm less than e in any given Sobolev space, provided the approximation is chosen 
appropriately Thus, we obtain 

By the assumption of the Lemma we have 

(ae + A^)a2 = a^2 + ^^02 + 
(here is a compact operator). Hence, 

did2 = 0^2 + Aea2 + a' + K^, 

where a' denotes the symbol (cr^ — (7i)(72 with a small norm. As we let £ ^ 0, this equality 
shows that the difference 

(7i(72 — CT1CT2 

is a norm limit of a family of compact operators K^. Therefore, the difference is compact 
as well. 

□ 

Lemma 3. The composition formula is valid for all symbols ai € Eq, a2 G E(M, tt). 
Moreover, the error term has order —1 in the Sobolev spaces. 

Proof. Obviously, it is sufficient to prove the formula when ai either corresponds to an 
operator with a smooth symbol or a family of pseudo differential operators in the fibers. 
We consider the two possibilities separately. 
1) Let 

(Ti = {a{x,y,^,ri),a{x,y,^,0)) 

correspond to a smooth symbol a{x,y,$,,rj) and (J2 = [pmiPx) be a general symbol. The 
composition of the corresponding operators has the form 

a{p + px -Pm))- 
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The desired composition formula will be proved if we show that 

apM = (34) 
and for a compact operator- valued symbol qx = Px ~ Pm ^ similar comparison is valid 

aqx = aqx. (35) 



a) To prove ( |5^ ) we have to show that the usual composition formula for ipDO^s on M 
remains valid in this case (since the expressions for the operators in (|3^) do not contain 
additional operator- valued components). 

The usual proof of the composition formula (e.g., see [|T^, p. 32) can be repeated 
verbatim to establish (|3^ . Moreover, the composition formula is valid modulo an operator 
of order —1 in the scale of Sobolev spaces. 

b) To prove (^) let us represent operator a as an operator on X with an operator- 
valued symbol (see [0]): 

a'(x,0=a(^x,i/,e,-^^) : L\Y.,) L\Y,). 

Let us first prove that the composition formula is satisfied modulo a compact operator in 
L'^{M). From the composition formula for operators with operator- valued symbols (see 
01) we obtain 



aqx - a'qx = 0. 



(36) 



On the other hand, the right-hand side of the desired formula ( pS]) in this case is equal to 

aqx- 

To prove that the two expressions a'qx and aqx differ by a compact operator, it 
suffices to show (see M) that the L^-norm of the compact operator- valued symbol 



qx{x,0 



(37) 



tends to zero as |^| — >• oo. Indeed, the symbol of the operator in the square brackets is 
equal to a{x, y, ^, rj) — a{x, y, ^, 0) and can be estimated as: 



\aix,y,^,v) - a(x,2/,C,0)| < C 



A similar estimate is valid for the derivatives in y (with a possibly different constant C). 

From this estimate it follows that the norm of the corresponding operator is estimated 
as follows 





r]\ 






\v 


1 + 1 



(^x, y, ^, -^^^ - 0) 



< 



c 



m(Y^)^HO{Y^) 



1 + 



26 



Hence, the norm of the symbol (P7| ) in is bounded by: 



d 



dy 

qx\\L2(Y^)^miY^) X 



< 



< 



C" 



m{Y^)^L^Y^) 



1 + 



Hence, it tends to zero as desired. This proves that the composition formula is valid in 
this case modulo a compact operator in L^(M). However, we need to prove a stronger 
statement that the error term has order —1 in the Sobolev scale. To prove this, it suffices 
to show that the difference in ( PB| ) and the operator with symbol { ^7\ j have order —1. 

First, similar to the proof of Theorem |l|, one can obtain for the symbol in ([37| ) the 
estimate of the form 



"A^+iei) 



s+l, 



L^{Y^)~^L^{Y^) 



< c 



and a similar estimate for its derivatives in x. Thus, similar to the Theorem 0, this proves 
that the symbol (a' — a)qx gives an operator of order —1 in the Sobolev scale. 

Second, to estimate the difference (|36D one should estimate the corresponding error 
term in the composition formula (e.g., see ||14[, p. 32). This can also be done. The details 
are left to the reader. 

2) Let us now verify the composition formula for a symbol 

CTi = {b{x,y,r]),B{x)) 

of a family of operators B{X) in the fibers and a2 = {pm,Px) as before. In this case 
the proof of the composition formula amounts to verifying the two comparisons (here 
qx =Px - Pm as before) 



Bq^ = Bqx, 



(38) 



BpM = bpM + Bpj^.^ - bpj^. (39) 

Concerning the first composition, it is easy to see that it is satisfied exactly. 

Let us establish comparison (^). To this end, we rewrite the terms BpM and bpj^ 
as operators on the base X with operator-valued symbols. We introduce the following 
notation. For a symbol a{x,y,^,ri) denote the operator-valued symbol a{x,y,^, —id/dy) 
by a' . Then (^) can be rewritten as 

Bp'm = {bpMY + qx, 
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where the operator symbol qx is equal to 

qx = Bp^ j - bpM- 



Thus, to prove that the comparison ( P^ ) is valid modulo a compact operator, it suffices 
to show that the operator-valued symbol 

Bp'm - {bpAiY - + bpM (40) 

is a compact operator for all (x,^) G T*X and its norm tends to zero as |^| — > oo. 

First of all, the compactness of (1^) is easy to obtain taking the symbols of the 
operators involved: 

a{Bp'^, - {bpM)' - B%j + h^i) = 0. (41) 

Here we omitted for brevity the variables x, y in the symbols. 

Let us now show that the norm of the symbol (|40D tends to zero as |.^| ^ oo. To this 
end, we rewrite this symbol in the form 

B{pm - Pm)' - {h{pM - Pm))' = {B - h'){pM - Pm)' + h'{pM - Pm)' - {h{pM - Pm))' ■ 

(42) 

For the difference Pm — Pm the following estimate can be obtained 

\PM{i,Tl) -Pm{^,V)\ < 





T]\ 






\v 


1 + 1 



Similar to the previous part of the proof this shows that the operator-valued symbol 
{B — h'){jpM — Pm)' in (0) tends to zero for ^ — > oo. Let us estimate the remaining term 

^'{Pm - Pm)' - (Kpm - Pm))' 

by means of the usual composition formula on the fiber. It is a pseudodifferential operator 
of order —1 with a symbol C{C,,ri) estimated as 

m,v)\< 



lei + 1^1 + 1 

Thus, the norm of the corresponding operator- valued symbol tends to zero as |^| oo. 
The proof that the composition formula is valid modulo an operator of order —1 can be 
obtained similar to the previous part of the proof. 

This completes the proof of the composition formula. 

□ 
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